Quality assurance (QA) refers to the process of checking to see whether a product or service meets expectations. It is desirable to test software products in order to identify and fix defects, bugs, and other problems before the products are deployed in a production or customer environment. For example, a backup application in a production environment that fails to properly backup data could be disastrous in the event that data needs to be restored. An e-commerce application that fails to properly process payments may lead to missed orders and lost revenue. If defects are reported, it is desirable to attempt to reproduce them and document the steps in order to trace the problem and retest the updated code.
Typically, test cases are created manually for a particular software system, product, service, or application. Once the manual test cases have been created, they can then be automated. Automating a test case from a manual test case allows a particular set of actions or operations to be performed against the system being tested automatically or with little or no user involvement. The conventional process to automate a test case, however, is very laborious: an automation engineer reviews each and every step of a previously created manual test case and develops a custom script. Efforts are often duplicated because in many cases a test case will include steps that overlap with the steps of another test case. Any changes or updates to a manual test case must be manually propagated to the corresponding automated script. It can be very difficult to keep these two in sync.
In some cases, a test case may not be automated because of the time required to create and maintain the corresponding automated test script. This can lead to increased development time because testing may need to be carried out manually. Another problem with traditional methods of testing is that exploratory or ad hoc testing scenarios are often not documented. That is, ad hoc or exploratory test cases are not usually documented in the test repository. Thus, testers may execute some non-documented test cases.
Executing non-documented test cases can cause discrepancies in the test metrics. The lack of a record can lead to uncertainty as to exactly what has been tested. Thus, testers often have to spend a good amount of time to document the exploratory test cases even though such test cases may be used sparingly. The dilemma is that a tester may spend a good amount of time automating the exploratory test cases but the time spent to automate may not be worth spending as they may not be used later or may be used relatively infrequently. Further, defects that are escalated can require a specific scenario to be tested. These test cases are often not documented in the test repository. In some cases, even though they are documented they are not executed as they require a specific scenario which could be a corner case or require a complex setup or a negative test case.
Therefore, there is a need for systems and techniques that provide for dynamic test case creation and documentation to the test repository through automation. It would be desirable to provide for the dynamic creation of a test case and running of it in automated way. Such systems and techniques can provide more confidence in the quality of the product and can reduce the number and severity of mistakes in the future releases as such issues can be easily tested in the current release.
The subject matter discussed in the background section should not be assumed to be prior art merely as a result of its mention in the background section. Similarly, a problem mentioned in the background section or associated with the subject matter of the background section should not be assumed to have been previously recognized in the prior art. The subject matter in the background section merely represents different approaches, which in and of themselves may also be inventions. EMC, Data Domain, Data Domain Restorer, and Data Domain Boost are trademarks of EMC Corporation.